System and method for determining a flushness of a fastener

ABSTRACT

A system for measuring a flushness of a fastener in a surface includes a plurality of lights configured to illuminate the fastener and the surface. The system also includes a camera configured to capture a plurality of images of the fastener and the surface while the lights sequentially illuminate the fastener and the surface. The system also includes a computing system configured to determine a normal map of the fastener and the surface based at least partially upon the images, determine a depth map of the fastener and the surface based at least partially upon the normal map, and determine the flushness of the fastener with respect to the surface based at least partially upon the depth map.

FIELD OF THE DISCLOSURE

The present disclosure is directed to systems and methods for measuringa flushness of a fastener. More particularly, the present disclosure isdirected to handheld systems and methods for measuring a flushness of afastener in a surface.

BACKGROUND

Two components are oftentimes held together by one or more fastenerssuch as screws, bolts, nails, etc. It may be desirable to have the(e.g., head of the) fastener be flush with the (e.g., outer) surface ofone of the two components. This may prevent the fastener frominadvertently contacting/snagging an object (e.g., a person) that ismoving with respect to the components and the fastener. In addition,when the components and fastener are part of a moving vehicle, such asan aircraft, having the fastener flush may reduce drag, thereby makingthe vehicle more aerodynamic.

Conventional systems and methods for measuring flushness of a fastenerinvolve a gauge that captures measurements physically/manually. Suchgauges have low repeatability, because even a slight pressure in anon-normal direction can alter the measurements. In addition, differentgauges may be needed to capture measurements for fasteners of differentsizes. Therefore, improved systems and methods for measuring a flushnessof a fastener in a surface are needed.

SUMMARY

A system for measuring a flushness of a fastener in a surface isdisclosed. The system includes a plurality of lights configured toilluminate the fastener and the surface. The system also includes acamera configured to capture a plurality of images of the fastener andthe surface while the lights sequentially illuminate the fastener andthe surface. The system also includes a computing system configured todetermine a normal map of the fastener and the surface based at leastpartially upon the images, determine a depth map of the fastener and thesurface based at least partially upon the normal map, and determine theflushness of the fastener with respect to the surface based at leastpartially upon the depth map.

A method for measuring a flushness of a fastener in a surface is alsodisclosed. The method includes positioning a system with respect to thefastener and the surface. The system includes a camera and a pluralityof lights. The method also includes sequentially illuminating thefastener and the surface with the lights. The method also includescapturing a plurality of images of the fastener and the surface whilethe lights sequentially illuminate the fastener and the surface. Themethod also includes determining a normal map of the fastener and thesurface based at least partially upon the images. The method alsoincludes determining a depth map of the fastener and the surface basedat least partially upon the normal map. The method also includesdetermining the flushness of the fastener with respect to the surfacebased at least partially upon the depth map.

In another embodiment, the method includes positioning a system withrespect to the fastener and the surface. The system includes a cameraand a plurality of lights. Positioning the system includes positioning aframe member of the system near, or in contact with, the surface suchthat a line of sight exists from the camera, through an opening in theframe member, and to the fastener. Positioning the system also includespositioning the system such that a distance between each light and thefastener is substantially constant. Positioning the system also includesorienting the system such that a slant angle between the camera and eachlight is substantially constant. The method also includes sequentiallyilluminating the fastener and the surface with the lights. The methodalso includes capturing a plurality of images of the fastener and thesurface, at least one image corresponding to each of the lightsilluminating the fastener and the surface. The method also includesdetermining a normal map of the fastener and the surface based at leastpartially upon the images. Determining the normal map includesdetermining a normal vector at one or more pixels in the images based atleast partially upon one or more light source vectors and an intensityof the one or more pixels in the images. The method also includesdetermining a depth map of the fastener and the surface by integratingthe normal map. The method also includes applying a high-pass filter tothe depth map to remove one or more dome-like features in the depth mapto produce a flattened depth map. The method also includes generating apoint cloud of the fastener and the surface based at least partiallyupon the flattened depth map. The method also includes determining theflushness of the fastener with respect to the surface based at leastpartially upon the point cloud.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the present teachings, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate aspects of the present teachingsand together with the description, serve to explain the principles ofthe present teachings.

FIGS. 1A and 1B illustrate perspective views of a system for measuring aflushness of a fastener in a surface, according to an implementation.FIG. 1C illustrates a side view of the system, according to animplementation.

FIG. 2 illustrates a flowchart of a method for measuring the flushnessof the fastener in the surface (e.g., using the system from FIG. 1),according to an implementation.

FIG. 3 illustrates a calibration panel to correct lens distortion of acamera in the system, according to an implementation.

FIGS. 4A-D illustrate images of a chrome sphere captured by the camera,according to an implementation.

FIG. 5 illustrates a graph showing source vector positions that arecalculated using the images from FIGS. 4A-D, according to animplementation.

FIG. 6 illustrates a schematic view of the system positioned withrespect to the fastener in the surface, according to an implementation.

FIG. 7A illustrates an image of the fastener and the component capturedby the system of FIG. 1 when a first light in the system is on (and theother lights are off). FIG. 7B illustrates an image of the fastener andthe component captured by the system of FIG. 1 when a second light inthe system is on (and the other lights are off). FIG. 7C illustrates animage of the fastener and the component captured by the system of FIG. 1when a third light in the system is on (and the other lights are off).FIG. 7D illustrates an image of the fastener and the component capturedby the system of FIG. 1 when a fourth light in the system is on (and theother lights are off).

FIG. 8 illustrates an image of the fastener and the component capturedby the system of FIG. 1 when all lights in the system are on, accordingto an implementation.

FIG. 9A illustrates a graph of a depth map in a substantially domeshape, according to an implementation. FIG. 9B illustrates a graph ofthe depth map after it has been flattened (e.g., using filtering),according to an implementation.

FIG. 10 illustrates a process for generating the depth map (e.g., usingfiltering), according to an implementation.

FIG. 11 illustrates a graph of the depth map after the process shown inFIG. 10, according to an implementation.

FIG. 12 illustrates an image of a 3D point cloud of the fastener and thesurrounding surface generated using the depth map from FIG. 11,according to an implementation.

FIG. 13 illustrates a schematic view of a computing system forperforming at least a portion of the method(s) disclosed herein,according to an implementation.

It should be noted that some details of the figures have been simplifiedand are drawn to facilitate understanding rather than to maintain strictstructural accuracy, detail, and scale.

DESCRIPTION

Reference will now be made in detail to the present teachings, examplesof which are illustrated in the accompanying drawings. In the drawings,like reference numerals have been used throughout to designate identicalelements. In the following description, reference is made to theaccompanying drawings that form a part thereof, and in which is shown byway of illustration specific examples of practicing the presentteachings. The following description is, therefore, merely exemplary.

The systems and methods disclosed herein may provide allow a user tomeasure a flushness of a fastener in a surface more quickly, moreaccurately, and more precisely. In addition, the system may be handheldfor easy movement/use by the user.

FIGS. 1A-1C illustrate views of a system 100 for measuring a flushnessof a fastener 180 in a surface 190, according to an implementation. Thefastener 180 may be or include a screw, a bold, a nail, etc. that isinserted into a corresponding hole formed in the surface 190. Thesurface 190 may be or include a component on a vehicle. For example, thesurface 190 may be or include an outer surface (i.e., skin) of anaircraft.

The system 100 may include a camera 110 and one or more lights 120A-D.The camera 110 may be stationary/fixed with respect to the object underinspection (e.g., the fastener 180 and/or the surface 190). Although thesystems and methods described herein use a single camera 110 andphotometric stereo, other systems and methods may also be used tomeasure the flushness of the fastener 180. Other such systems andmethods may include stereo vision, random pattern generation (stereo),structured light scanning (SLS), and shape from shadow (single imagesurface generation).

The lights 120A-D may be or include light-emitting diodes (LEDs) havingsubstantially identical intensities. The lights 120A-D may becircumferentially-offset from one another (e.g., by equalamounts/angles) with respect to a central longitudinal axis 112 thatextends through the camera 110 and/or the fastener 180. For example,when four lights 120A-D are used, the lights 120A-D may be offset fromone another by about 90°. Although the implementation described hereinincludes four lights 120A-D, other implementations may include three ormore lights.

The system 100 may also include one or more arms (four are shown 130A-D)that are positioned at least partially between the camera 110 and/or thelights 120A-D on one side and the fastener 180 and/or the surface 190 onthe other side. As shown, the arms 130A-D may becircumferentially-offset from one another about the central longitudinalaxis 112. A frame member 140 may be coupled to distal (e.g., lower) endsof the arms 130A-D. The frame member 140 may define an opening 142therethrough, and the camera 110 may be able to view the fastener 180through the opening 142.

FIG. 2 illustrates a flowchart of a method 200 for measuring theflushness of the fastener 180 in the surface 190 (e.g., using the system100 from FIG. 1), according to an implementation. In at least oneimplementation, the method 200 may include calibrating the camera 110,as at 202. The camera 110 may be a pin-hole camera, which may have lensdistortion that causes the images captured by the camera to lead toinaccurate measurements. An open CV camera calibration module may beused to correct the distortion. More particularly, the calibrationmodule may include a calibration panel (e.g., a dot pattern board) 300,as shown in FIG. 3, to correct for the distortion. Correcting for thedistortion may include calculating the distortion coefficients andcamera extrinsic parameters from/using the calibration module, and usingthese coefficients and parameters to reduce the distortion in theimages.

The method 200 may also include determining light source vectors, as at204. The light source vectors may be determined before or after thecamera 110 is calibrated. Determining the light source vectors mayinclude positioning the system (e.g., the camera 110 and/or the lights120A-D) with respect to a metallic and/or reflective (e.g., chrome)sphere. The sphere may be positioned at the same distance from thecamera 110 and/or the lights 120A-D as the fastener 180 willsubsequently be positioned. The user may ensure that there is nobackground lighting or reflecting surfaces (other than the sphere) inthe camera's field of view. The user may then sequentially turn on thelights 120A-D, one at a time, and capture an image of the sphere withthe camera 110 when the sphere is illuminated by each light 120A-D. Thismay result in four reflective images 410, 420, 430, 440, as shown inFIGS. 4A-D.

The light source vectors may then be calculated based at least partiallyupon the reflective images 410, 420, 430, 440. More particularly, thelight source vectors may be calculated using the reflections in theimages 410, 420, 430, 440 and the known center of the sphere. FIG. 5illustrates a graph 500 showing the light source vectors, according toan implementation.

The method 200 may also include positioning the system 100 (e.g., thecamera 110 and/or the lights 120A-D) with respect to the fastener 180and/or the surface 190, as at 206. This may occur after the camera 110is calibrated and/or the light source vectors are determined.

Positioning the system 100 with respect to the fastener 180 and/or thesurface 190 may include placing the frame member 140 near, or in contactwith, the surface 190 such that a line of sight exists from the camera110, through the opening 142 in the frame member 140, to the fastener180. As used in the preceding sentence, “near” may refer to 10 cm orless, 5 cm or less, or 1 cm or less. The distance between the lights120A-D and the fastener 180 may be substantially constant to ensure thatfastener 180 and the surrounding area of the surface 190 are illuminatedsufficiently and uniformly. As used in the preceding sentence,“substantially constant” may refer to within about 5 mm or less, withinabout 3 mm or less, or within about 1 mm or less. In at least oneimplementation, the camera 110, the fastener 180, and the opening 142may share the central longitudinal axis 112 (i.e., they may all beco-aligned).

In addition, positioning the system 100 with respect to the fastener 180and/or the surface 190 may include orienting the system 100 such that aslant angle σ between the camera 110 and each light 120A-D (with thefastener 180 as the vertex) is substantially constant, as shown in FIG.6. As used in the preceding sentence, “substantially constant” may referto about 5° or less, about 3° or less, or about 1° or less. For example,the slant angle σ of each camera 120A-D may be from about 20° to about70°, about 30° to about 60°, about 40° to about 50°. For example, theslant angle σ may be about 45° for each light 120A-D.

As mentioned above, an offset angle α between each adjacent pair oflights 120A-D (with the central longitudinal axis 112 as the vertex) maybe substantially constant, as shown in FIG. 6. For example, the offsetangle α may be 90° when the system 100 includes four lights 120A-D.

The method 200 may also include capturing a plurality of (e.g., four)images while the lights 120A-D sequentially illuminate the fastener 180(and the surrounding area of the surface 190), as at 208. FIG. 7Aillustrates an image 710 of the fastener 180 (and the surrounding areaof the surface 190) illuminated by the first light 120A. The first light120A may then be turned off, and the second light 120B may be turned on.FIG. 7B illustrates an image 720 of the fastener 180 (and thesurrounding area of the surface 190) illuminated by the second light120B. The second light 120B may then be turned off, and the third light120C may be turned on. FIG. 7C illustrates an image 730 of the fastener180 (and the surrounding area of the surface 190) illuminated by thethird light 120C. The third light 120C may then be turned off, and thefourth light 120D may be turned on. FIG. 7D illustrates an image 740 ofthe fastener 180 (and the surrounding area of the surface 190)illuminated by the fourth light 120D.

In at least one implementation, the method 200 may also includecapturing a fifth image while all of the lights 120A-D illuminate thefastener 180 (and the surrounding area of the surface 190)simultaneously, as at 210. FIG. 8 illustrates the fifth image 750. Thefifth image 750 may be used to determine a diameter of the fastener 180.

In some instances, one or more of the raw images 710, 720, 730, 740,and/or 750 may be marred with noise. Accordingly, the method 200 mayalso include filtering the images 710, 720, 730, 740, and/or 750, as at212. More particularly, bilateral filtering may be used to remove thenoise in low-frequency areas of the images 710, 720, 730, 740, and/or750 while retaining sharpness at the edges. The filter may be or includea double Gaussian filter that applies to intensity and distance.

The method 200 may also include calculating/determining a normal map(e.g., including x and y gradients) of the fastener 180 (and thesurrounding area of the surface 190) based at least partially upon theimages 710, 720, 730, 740, and/or 750, as at 214. In at least oneimplementation, the normal map may be calculated/determined using theimages 710, 720, 730, 740, but not the image 750. The normal map mayallow the orientation of the camera 110 and the lights 120A-D to be usedfor the scan calculation (described below) and the eventual extrusioninto a three-dimensional (3D) surface). The normal map may also be usedfor one or more of the calculations below, as the surface gradient is aprerequisite for the overall calculation of the algorithm (e.g., tocalculate a height of the fastener 180).

Surface gradients can be calculated from the normal vectors as definedin Equations 1-3 below:

$\begin{matrix}{\frac{\partial f}{\partial x} = \frac{N_{x}}{N_{z}}} & {{Equation}\mspace{14mu}(1)} \\{\frac{\partial f}{\partial y} = \frac{N_{y}}{N_{z}}} & {{Equation}\mspace{14mu}(2)}\end{matrix}$

Here, f represents the depth map of the fastener 180 and/or the surface190. The variables x, y, and Z represent the Cartesian coordinate axes.The variables N_(x), N_(y), and N_(z) represent the normal vectors alongthe X, Y, and Z axes, respectively. The terms

$\frac{\partial f}{\partial x}\mspace{14mu}{and}\mspace{14mu}\frac{\partial f}{\partial y}$represent the surface gradients along the depth map along the X and Zdirections. Equations 1-3 explain how to calculate real world surfacegradients of the depth map function f using normal calculated from the2D images.

$\begin{matrix}{{f\left( {x,y} \right)} = {{\oint_{C}{\left( {\frac{\partial f}{\partial x},\frac{\partial f}{\partial y}} \right) \cdot {dl}}} + c}} & \left( {{Equation}\mspace{14mu} 3} \right)\end{matrix}$

Equation 3 describes how to calculate a two-dimensional (2D) function inx and y given its partial derivatives along both axes. The symbol

indicates a contour integral, and the 2^(nd) term indicates a constant.Direct integration of Equation 3 may be iterative and slow. As a result,the function may be calculated in a closed form instead. This may bedone by transforming this as a minimization problem and solving thisminimization in the Fourier Domain to obtain a closed form solution.

In at least one implementation, the function f may be solved such thatthe mean square error in Equation 4 below is minimized.∫∫(|{tilde over (ƒ)}_(x)−{circumflex over (ƒ)}_(x)|²+|{tilde over(ƒ)}_(y)−{circumflex over (ƒ)}_(y)|²)dxdy   (Equation 4)In this equation, ƒ represents the original depth map that is beingevaluated. The variables {circumflex over (ƒ)}_(x) and {circumflex over(ƒ)}_(y) represent the estimated x and y gradients of ƒ, respectively,and the variables {tilde over (ƒ)}_(x) and {tilde over (ƒ)}_(y)represesnt the x and y gradients of the optimized depth function {tildeover (ƒ)}. The variable {tilde over (ƒ)} approximates the original depthmap function ƒ.

By transforming this problem into the Fourier domain and solving for{tilde over (ƒ)}, Equation 5 may be obtained.

$\begin{matrix}{{\overset{\sim}{F}(\omega)} = \frac{{{- j}\;\omega_{x}{{\hat{F}}_{x}(\omega)}} - {j\;\omega_{y}{{\hat{F}}_{y}(\omega)}}}{\omega_{x}^{2} + \omega_{y}^{2}}} & \left( {{Equation}\mspace{14mu} 5} \right)\end{matrix}$

In Equation 5, {tilde over (F)}(ω) denotes the Fourier Transform of thedepth map function {tilde over (ƒ)}. The desired depth map {tilde over(ƒ)} is obtained by calculating the inverse Fourier Transform of {tildeover (F)}(ω).

A normal vector at each pixel in the images 710, 720, 730,740 may bedetermined based at least partially upon the light vectors (e.g., fromstep 204) and the intensity of the pixels in the images 710, 720, 730,740. For example, the normal vectors may be determined using Equation 4below.I _(n)(x,y)=L _(n)·g _(n)(x,y)   (Equation 6)

The pixel intensity of the nth image I_(n) at pixel x, y is a dotproduct of the albedo vector g at the same pixel and the light vector Lfor that image. With at least three measurements at each pixel, and theknown light intensities, the normal vector may be calculated by solvinga linear system of equations at each pixel, as shown in Equations 5-7below.

$\begin{matrix}{\begin{pmatrix}{I_{1}\left( {x,y} \right)} \\{I_{2}\left( {x,y} \right)} \\. \\. \\{I_{n}\left( {x,y} \right)}\end{pmatrix} = {\begin{bmatrix}L_{11} & L_{12} & L_{13} \\L_{21} & L_{22} & L_{23} \\. & . & . \\. & . & . \\L_{n\; 1} & L_{n\; 2} & L_{n\; 3}\end{bmatrix}\begin{pmatrix}{g_{x}\left( {x,y} \right)} \\{g_{y}\left( {x,y} \right)} \\{g_{z}\left( {x,y} \right)}\end{pmatrix}}} & \left( {{Equation}\mspace{14mu} 7} \right) \\{{{N_{x}\left( {x,y} \right)} = \frac{g_{x}\left( {x,y} \right)}{{g\left( {x,y} \right)}}},{{N_{y}\left( {x,y} \right)} = \frac{g_{y}\left( {x,y} \right)}{{g\left( {x,y} \right)}}},{{N_{z}\left( {x,y} \right)} = \frac{g_{z}\left( {x,y} \right)}{{g\left( {x,y} \right)}}}} & \left( {{Equation}\mspace{14mu} 8} \right) \\{{{g\left( {x,y} \right)}} = \sqrt{{g_{x}\left( {x,y} \right)}^{2} + {g_{y}\left( {x,y} \right)}^{2} + {g_{z}\left( {x,y} \right)}^{2}}} & \left( {{Equation}\mspace{14mu} 9} \right)\end{matrix}$

In these equations, N_(x), N_(y), and N_(z) represent the normal vectorsalong the X, Y, and Z axes, respectively. The term |g(x,y)| indicatesthe magnitude of the albedo vector g.

The method 200 may also include calculating/determining a depth map ofthe fastener 180 (and the surrounding area of the surface 190) based atleast partially upon the normal map, as at 216. More particularly, thedepth map may be calculated/determined by integrating the normal map.The depth map may be used to determine the flushness of the fastener 180in the surface 190. The data initially reconstructs as a dome, which iswhy the flattening may be helpful.

FIG. 9A illustrates a graph of the depth map 910, according to animplementation. The reconstructed depth map 910 may have a significantlow-frequency component (e.g., the dome-shape) in one or two axesbecause the low frequencies cannot be exactly reconstructed due to thedenominator in Equation 5 becoming zero. The denominator ω_(x) ²+ω_(y) ²becomes zero when ω_(x) (frequency along x axis) and ω_(y) (frequencyalong y axis) is zero. Hence, the user may have to approximate the zerofrequency with a small value so as not to make the denominator zero.This is the reason that the zero frequency components resulting in thedome shape seen in FIG. 9A cannot be exactly reconstructed. A high-passfilter may be applied to the depth map 910 to remove the dome-likefeatures to produce a flat depth map 920, as shown in FIG. 9B.

The method 200 may also include modifying the (flat) depth map 920, asat 218. This is illustrated in FIG. 10. A Fourier transform of the flatdepth map 1010 may be combined with a filter response for high-passfiltering 1020 to produce a Fourier transform of a high-pass filtereddepth map 1030. The high-pass filtered depth map 1030 may be moreaccurate than the flat depth map 920.

The method 200 may also include generating an image of the depth map1100 based at least partially upon the Fourier transform of thehigh-pass filtered depth map 1030, as at 220. This is shown in FIG. 11.The outer ring is the surface 190 to which the fastener 180 isattaching. The height of the surface 190 minus the height of thefastener 180 may be used to determine the flushness.

The method 200 may also include generating a point cloud of the fastener180 (and the surrounding area of the surface 190) based at leastpartially upon the depth map 1100, as at 222. The point cloud may be theearliest form of data returned from the calculation. More particularly,the point cloud is a return of the pixel count from the image board ofthe camera 110. The cloud to 3D coordinate pixel may be xy, and thepoint cloud is xyz in a Cartesian coordinate system, which provides theheight z. In addition, the point cloud may be used to represent thedepth map. Using this technique also allows the user to speed up theimage processing rather than creating large matrices and (e.g.,triangular) meshes, as is done in conventional systems and methods, andwhich hinders the speed and overall performance.

FIG. 12 illustrates an image of a 3D point cloud 1200 of the fastener180 and the surrounding surface 190 generated using the depth map 1100from FIG. 11, according to an implementation. The point cloud 1200 mayinclude an inner fastener region and an outer fastener region.

The method 200 may also include measuring/determining the flushness ofthe fastener 180 with respect to the surrounding surface 190 based atleast partially upon the depth map 1100 and/or the point cloud 1200, asat 224. More particularly, this may include measuring the flushness ofthe fastener 180 in one or more (e.g., four) regions around thecircumference of the fastener 180 (e.g., North, East, South, and West).Each region may be (primarily) be illuminated by a different one of thelights 120A-D.

The method 200 may also include modifying the size/dimensions of thefastener 180 and/or the hole into which the fastener 180 is inserted inresponse to the flushness of the fastener 180 with respect to thesurrounding surface 190, as at 226. This modification may be done to thefastener 180 and/or hole upon which the method 200 is performed, and/orto subsequent fasteners 180 and/or holes. The modification may improvethe flushness.

The measurement repeatability using the method 200 is +/−0.0005 inchesor less. The accuracy of the method 200 is +/−0.0035 inches or less. Theprocessing time (e.g., to perform steps 212-224 of the method 200) is 5seconds or less per fastener 180.

In some implementations, the method(s) 200 of the present disclosure maybe executed by a computing system. FIG. 13 illustrates an example ofsuch a computing system 1300, in accordance with some implementations.The computing system 1300 may include a computer or computer system1301A, which may be an individual computer system 1301A or anarrangement of distributed computer systems. The computer system 1301Aincludes one or more analysis modules 1302 that are configured toperform various tasks according to some implementations, such as one ormore methods disclosed herein. To perform these various tasks, theanalysis module 1302 executes independently, or in coordination with,one or more processors 1304, which is (or are) connected to one or morestorage media 1306. The processor(s) 1304 is (or are) also connected toa network interface 1307 to allow the computer system 1301A tocommunicate over a data network 1309 with one or more additionalcomputer systems and/or computing systems, such as 1301B, 1301C, and/or1301D (note that computer systems 1301B, 1301C and/or 1301D may or maynot share the same architecture as computer system 1301A, and may belocated in different physical locations, e.g., computer systems 1301Aand 1301B may be located in a processing facility, while incommunication with one or more computer systems such as 1301C and/or1301D that are located in one or more data centers, and/or located invarying countries on different continents).

A processor may include a microprocessor, microcontroller, processormodule or subsystem, programmable integrated circuit, programmable gatearray, or another control or computing device.

The storage media 1306 may be implemented as one or morecomputer-readable or machine-readable storage media. Note that while inthe example implementation of FIG. 13 storage media 1306 is depicted aswithin computer system 1301A, in some implementations, storage media1306 may be distributed within and/or across multiple internal and/orexternal enclosures of computing system 1301A and/or additionalcomputing systems. Storage media 1306 may include one or more differentforms of memory including semiconductor memory devices such as dynamicor static random access memories (DRAMs or SRAMs), erasable andprogrammable read-only memories (EPROMs), electrically erasable andprogrammable read-only memories (EEPROMs) and flash memories, magneticdisks such as fixed, floppy and removable disks, other magnetic mediaincluding tape, optical media such as compact disks (CDs) or digitalvideo disks (DVDs), BLURAY® disks, or other types of optical storage, orother types of storage devices. Note that the instructions discussedabove may be provided on one computer-readable or machine-readablestorage medium, or may be provided on multiple computer-readable ormachine-readable storage media distributed in a large system havingpossibly plural nodes. Such computer-readable or machine-readablestorage medium or media is (are) considered to be part of an article (orarticle of manufacture). An article or article of manufacture may referto any manufactured single component or multiple components. The storagemedium or media may be located either in the machine running themachine-readable instructions, or located at a remote site from whichmachine-readable instructions may be downloaded over a network forexecution.

In some implementations, computing system 1300 contains one or moreflushness determination module(s) 1308. In the example of computingsystem 1300, computer system 1301A includes the flushness determinationmodule 1308. In some implementations, a single flushness determinationmodule may be used to perform some aspects of one or moreimplementations of the methods disclosed herein. In otherimplementations, a plurality of flushness determination modules may beused to perform some aspects of methods herein.

It should be appreciated that computing system 1300 is merely oneexample of a computing system, and that computing system 1300 may havemore or fewer components than shown, may combine additional componentsnot depicted in the example implementation of FIG. 13, and/or computingsystem 1300 may have a different configuration or arrangement of thecomponents depicted in FIG. 13. The various components shown in FIG. 13may be implemented in hardware, software, or a combination of bothhardware and software, including one or more signal processing and/orapplication specific integrated circuits.

Further, the steps in the processing methods described herein may beimplemented by running one or more functional modules in informationprocessing apparatus such as general purpose processors or applicationspecific chips, such as ASICs, FPGAs, PLDs, or other appropriatedevices. These modules, combinations of these modules, and/or theircombination with general hardware are included within the scope of thepresent disclosure.

Computational interpretations, models, and/or other interpretation aidsmay be refined in an iterative fashion; this concept is applicable tothe methods discussed herein. This may include use of feedback loopsexecuted on an algorithmic basis, such as at a computing device (e.g.,computing system 1300, FIG. 13), and/or through manual control by a userwho may make determinations regarding whether a given step, action,template, model, or set of curves has become sufficiently accurate forthe evaluation of the subsurface three-dimensional geologic formationunder consideration.

Notwithstanding that the numerical ranges and parameters setting forththe broad scope of the disclosure are approximations, the numericalvalues set forth in the specific examples are reported as precisely aspossible. Any numerical value, however, inherently contains certainerrors necessarily resulting from the standard deviation found in theirrespective testing measurements. Moreover, all ranges disclosed hereinare to be understood to encompass any and all sub-ranges subsumedtherein.

While the present teachings have been illustrated with respect to one ormore implementations, alterations and/or modifications can be made tothe illustrated examples without departing from the spirit and scope ofthe appended claims. In addition, while a particular feature of thepresent teachings may have been disclosed with respect to only one ofseveral implementations, such feature may be combined with one or moreother features of the other implementations as may be desired andadvantageous for any given or particular function. As used herein, theterm “at least one of A and B” with respect to a listing of items suchas, for example, A and B, means A alone, B alone, or A and B. Thoseskilled in the art will recognize that these and other variations arepossible. Furthermore, to the extent that the terms “including,”“includes,” “having,” “has,” “with,” or variants thereof are used ineither the detailed description and the claims, such terms are intendedto be inclusive in a manner similar to the term “comprising.” Further,in the discussion and claims herein, the term “about” indicates that thevalue listed may be somewhat altered, as long as the alteration does notresult in nonconformance of the process or structure to the intendedpurpose described herein. Finally, “exemplary” indicates the descriptionis used as an example, rather than implying that it is an ideal.

It will be appreciated that variants of the above-disclosed and otherfeatures and functions, or alternatives thereof, may be combined intomany other different systems or applications. Various presentlyunforeseen or unanticipated alternatives, modifications, variations, orimprovements therein may be subsequently made by those skilled in theart which are also intended to be encompasses by the following claims.

What is claimed is:
 1. A system for measuring a flushness of a fastenerin a surface, comprising: a plurality of lights configured to illuminatethe fastener and the surface; a camera configured to capture a pluralityof images of the fastener and the surface while the lights sequentiallyilluminate the fastener and the surface; and a computing systemconfigured to: determine a normal map of the fastener and the surfacebased at least partially upon the images; determine a depth map of thefastener and the surface based at least partially upon the normal map;and determine the flushness of the fastener with respect to the surfacebased at least partially upon the depth map.
 2. The system of claim 1,wherein the system further comprises a plurality of arm members that arepositioned at least partially between the camera and the fastener, andwherein the arm members are circumferentially-offset from one anotheraround a central longitudinal axis through the camera, the fastener, orboth.
 3. The system of claim 2, wherein the system further comprises aframe member coupled to or integral with the arm members, and whereinthe frame member defines an opening through which the camera isconfigured to view the fastener.
 4. The system of claim 1, wherein thelights are circumferentially-offset from one another around a centrallongitudinal axis through the camera, and wherein a distance betweeneach light and the fastener is substantially constant when the imagesare captured.
 5. The system of claim 1, wherein an offset angle betweeneach adjacent pair of lights is substantially constant, and wherein aslant angle of each light is from about 30° to about 60° when the imagesare captured.
 6. A method for measuring a flushness of a fastener in asurface, comprising: positioning a system with respect to the fastenerand the surface, wherein the system comprises a camera and a pluralityof lights; sequentially illuminating the fastener and the surface withthe lights; capturing a plurality of images of the fastener and thesurface while the lights sequentially illuminate the fastener and thesurface; determining a normal map of the fastener and the surface basedat least partially upon the images; determining a depth map of thefastener and the surface based at least partially upon the normal map;and determining the flushness of the fastener with respect to thesurface based at least partially upon the depth map.
 7. The method ofclaim 6, further comprising calibrating the camera using a dot patternboard to reduce lens distortion of the camera, wherein the camera iscalibrated before the images are captured.
 8. The method of claim 6,wherein positioning the system comprises positioning a frame member ofthe system near, or in contact with, the surface such that a line ofsight exists from the camera, through an opening in the frame member,and to the fastener.
 9. The method of claim 8, wherein positioning thesystem comprises orienting the system such that: a slant angle betweenthe camera and each light is substantially constant; and a distancebetween each light and the fastener is substantially constant.
 10. Themethod of claim 6, further comprising applying a double Gaussian filterto the images, wherein the filter applies to intensity and distance. 11.The method of claim 6, further comprising: sequentially illuminating areflective object with the lights; capturing a plurality of reflectiveimages while the lights sequentially illuminate the reflective object;and determining a plurality of light source vectors based at leastpartially upon the reflective images.
 12. The method of claim 11,wherein the reflective object comprises a chrome sphere.
 13. The methodof claim 12, wherein the light source vectors are determined based atleast partially upon reflections in the reflective images and a knowncenter of the sphere.
 14. The method of claim 11, wherein determiningthe normal map comprises determining a normal vector at one or morepixels in the images based at least partially upon the light sourcevectors and an intensity of the one or more pixels in the images. 15.The method of claim 6, wherein determining the normal map comprisesdetermining x and y gradients.
 16. The method of claim 6, whereindetermining the depth map comprises integrating the normal map.
 17. Themethod of claim 6, wherein the depth map comprises one or morelow-frequency features that appear dome-shaped, and further comprisingapplying a high-pass filter to the depth map to remove the one or morelow-frequency features to produce a flattened depth map.
 18. The methodof claim 6, further comprising generating a point cloud of the fastenerand the surface based at least partially upon the depth map, wherein theflushness of the fastener is determined based at least partially uponthe point cloud.
 19. The method of claim 6, further comprising modifyingthe fastener, or an opening in the surface through which the fastener isinserted, in response to the flushness of the fastener.
 20. A method formeasuring a flushness of a fastener in a surface, comprising:positioning a system with respect to the fastener and the surface,wherein the system comprises a camera and a plurality of lights, andwherein positioning the system comprises: positioning a frame member ofthe system near, or in contact with, the surface such that a line ofsight exists from the camera, through an opening in the frame member,and to the fastener; positioning the system such that a distance betweeneach light and the fastener is substantially constant; and orienting thesystem such that a slant angle between the camera and each light issubstantially constant; sequentially illuminating the fastener and thesurface with the lights; capturing a plurality of images of the fastenerand the surface, at least one image corresponding to each of the lightsilluminating the fastener and the surface; determining a normal map ofthe fastener and the surface based at least partially upon the images,wherein determining the normal map comprises determining a normal vectorat one or more pixels in the images based at least partially upon one ormore light source vectors and an intensity of the one or more pixels inthe images; determining a depth map of the fastener and the surface byintegrating the normal map; applying a high-pass filter to the depth mapto remove one or more dome-like features in the depth map to produce aflattened depth map; generating a point cloud of the fastener and thesurface based at least partially upon the flattened depth map; anddetermining the flushness of the fastener with respect to the surfacebased at least partially upon the point cloud.